Analyzing operational results using rule-based policy models

ABSTRACT

Methods, systems, and products that execute modeled business controls on a modeled input work item to generate a corresponding modeled output work item. Input data values included in the modeled input work item either match or approximate corresponding values included in a corresponding actual input work item. Each modeled business control is defined by business rules that represent a business policy, and the business rules and the input data values of the modeled input work item are used to generate the modeled output work item. An actual output work item from a business process is compared to the modeled output work item. The actual output work item includes actual output data values. The business process changes the actual input work item into the actual output work item. The modeled output work item includes modeled output data values. Differences between the actual and modeled output work items are reported.

BACKGROUND

The present invention relates to the rule-based modeling of policy, andthe use of modeled policy to analyze operational results.

Operational results are the outputs produced by business processes(e.g., bills produced by a billing process; payments produced by aclaims process; reports produced by a financial reporting process). Inperforming processes, people and systems transform inputs intooperational results.

Processes are generally governed by business policies that define theset of rules used to transform inputs into operational results. Policiescan derive from many sources, including management (e.g., bestpractices), trading partners (e.g., contracts and service levelagreements), and regulatory bodies (e.g., statutes). Adherence to policyis necessary to ensure optimal business outcomes and/or to comply withcontract or law.

Organizations sometimes have trouble clearly defining and enforcingpolicy, and thus are subject to suboptimal operational results and/orrisk of non-compliance. In addition, organizations face the challenge ofchanging policies and understanding the impact of policy change. Thisincludes the need to keep existing policies current as well as tocontinuously design new, better policies.

SUMMARY

This specification describes systems and processes used to analyzeoperational (that is, actual) results using modeled policy and toquantitatively measure discrepancies between the actual results andresults predicted by the modeled policy.

In one aspect, the invention features a system that includes an analysisengine which compares one or more actual output work items from anactual business process to one or more modeled output work items from amodeled business process and reports differences between them. Eachactual output work item includes one or more actual output data values,and the actual business process changes an actual input work item intothe actual output work item. Each modeled output work item includes oneor more modeled output data values, and the modeled business processoperates on one or more modeled input work items and generates the oneor more modeled output work items. Input data values included in eachmodeled input work item either match or approximate corresponding valuesincluded in a corresponding actual input work item. The modeled businessprocess includes one or more modeled business controls, each of whichcorresponds to an actual business control in the actual businessprocess. Each modeled business control is defined by a set of businessrules that represent a business policy. The analysis engine uses the setof business rules to generate the one or more modeled output work itemsusing the input data values of the one or more modeled input work items.

Particular implementations can include one or more of the followingfeatures. The analysis engine creates a rule audit trail that records,for an individual modeled output work item, a sequence of modeledbusiness controls that executed on an individual modeled input workitem. The rule audit trail also records a sequence of associatedbusiness rules that fired within the modeled business controls togenerate the individual modeled output work item. The rule audit trailalso records a set of data values input to each business rule that firedand a set of data values output from each business rule that fired. Theanalysis engine uses the rule audit trail to identify one or more rootcauses for any difference between the individual modeled output workitem and a corresponding actual output work item, where each root causeis a business rule. The analysis engine compares the one or more actualoutput work items and the one or more modeled output work items in nearreal-time and generates an alert if a difference is found. The analysisengine aggregates results of comparing the one or more actual outputwork items and the one or more modeled output work items to report thedifferences.

The analysis engine generates an alert when a discrepant work itemoccurs. The analysis engine compares the one or more actual output workitems to one or more updated modeled output work items from one or moreupdated modeled business controls and reports a change caused by usingthe one or more updated modeled business controls in place of the one ormore modeled business controls. An analytical data storage databasestores each modeled input work item and each modeled output work item.The analytical data storage database is constructed automatically basedupon a data model used to define the one or more modeled businesscontrols. A data model is used to construct the one or more modeledbusiness controls, and the data model is constructed automatically basedupon an operational data model used within an operational data storagethat stores each actual output work item and each actual input workitem.

The analysis engine is operable to generate an alert if the set ofbusiness rules that defines one of the modeled business controls isincomplete or inconsistent. The analysis engine generates an alert ifone of the actual output work items does not include any actual outputdata value that corresponds to a modeled output data value included inone of the modeled output work items.

In another aspect, the invention features a system that includes meansfor comparing one or more actual output work items from an actualbusiness process to one or more modeled output work items from a modeledbusiness process. The system also includes means for reportingdifferences between the one or more actual output work items and the oneor more modeled output work items. Each actual output work item includesone or more actual output data values, and the actual business processchanges an actual input work item into the actual output work item. Eachmodeled output work item includes one or more modeled output datavalues, and the modeled business process operates on a modeled inputwork item and generates a corresponding modeled output work item. Inputdata values included in each modeled input work item either match orapproximate corresponding values included in a corresponding actualinput work item. The modeled business process includes one or moremodeled business controls, each of which corresponds to an actualbusiness control in the actual business process. Each modeled businesscontrol is defined by a set of business rules that represent a businesspolicy, and the means for comparing uses the set of business rules togenerate the modeled output work item using the input data values of themodeled input work item.

In yet another aspect, the invention features a method and a computerprogram product that executes a modeled business process consisting ofone or more modeled business controls on a modeled input work item togenerate a corresponding modeled output work item. Input data valuesincluded in the modeled input work item either match or approximatecorresponding values included in a corresponding actual input work item.Each modeled business control is defined by a set of business rules thatrepresent a business policy, and executing the one or more modeledbusiness controls includes using the set of business rules and the inputdata values of the modeled input work item to generate the modeledoutput work item. An actual output work item from an actual businessprocess is compared to the modeled output work item. The actual outputwork item includes one or more actual output data values. The actualbusiness process changes the actual input work item into the actualoutput work item. The modeled output work item includes one or moremodeled output data values. Differences between the actual output workitem and the modeled output work item are reported.

Particular implementations can include one or more of the followingfeatures. A rule audit trail is created that records, for the modeledoutput work item, a sequence of the modeled business controls thatexecuted on the modeled input work item and a sequence of the businessrules that fired within the modeled business controls to generate themodeled output work item. A set of data values input to each businessrule that fired and a set of data values output from each business rulethat fired are recorded. The rule audit trail is used to identify one ormore root causes, where each root cause is a business rule that causes adifference between the individual modeled output work item and theactual output work item. The actual output work item is compared to anupdated modeled output work item from one or more updated modeledbusiness controls, and a change caused by using the one or more updatedmodeled business controls in place of the one or more modeled businesscontrols is reported. The set of business rules is created by modelinglogic of the business policy.

In another aspect, the invention features a system for comparing outputsof internal business controls to outputs of business controls modeling apolicy. An analysis engine compares one or more actual output work itemsfrom one or more actual business controls in an actual business processto one or more modeled output work items from one or more modeledbusiness controls. Each actual output work item includes a set ofvariables, where each variable stores one or more values. The one ormore actual business controls execute on and change an actual input workitem into the actual output work item. Each modeled output work itemincludes a set of variables, where each variable stores one or morevalues. The one or more modeled business controls execute on and changea modeled input work item into the modeled output work item. Each actualbusiness control is executed by an existing system or manually by one ormore people, and each modeled business control is executed by a ruleengine. Each modeled business control includes a set of business rulesthat describe a desired logic for the modeled business control, and theset of business rules transform the modeled input work item into themodeled output work item. Each business rule in the set of businessrules is associated with a policy source and a motivation, and eachmodeled input work item is extracted from operational data stores thatstore the one or more actual output work items. Each modeled input workitem is either a copy of a corresponding actual input work item or anapproximation of the corresponding actual input work item. The one ormore modeled business controls each fire one or more business rules fromthe set of business rules included in the respective modeled businesscontrol. The analysis engine creates a rule audit trail and stores therule audit trail in an analytical data store, where the rule audit trailrecords, for an individual modeled output work item, a sequence ofmodeled business controls and associated business rules that firedagainst a corresponding modeled input work item, a set of variableinputs to each business rule that fired, and a set of variable outputsfrom each business rule that fired. The analysis engine uses the ruleaudit trails to identify any differences between the actual output workitems and corresponding modeled output work items. For any differencebetween the actual output work items and the corresponding modeledoutput work items, the analysis engine uses the rule audit trails toidentify a root cause and generates reports.

Particular implementations can include one or more of the followingfeatures. The desired logic for each modeled business control is derivedfrom the policy, where the policy is an existing policy. Any differencesbetween the actual output work items and the corresponding modeledoutput work items are discrepancies between the policy and an executionof the policy. Alternatively, the desired logic for each modeledbusiness control is derived from the policy, where the policy is aproposed new policy. Any differences between the actual output workitems and the corresponding modeled output work items reflect theeffects of changing from an existing policy to the proposed new policy.

The systems and processes help organizations understand their existingpolicies and assess whether their operational results, and the processesused to generate their operational results, are in compliance with theirexisting policies. In addition, the systems and processes can be used toanalyze the expected impact of policy changes on operational results.The systems and processes can be used to analyze the impact of changingmarket conditions on operational results, in consideration of current ornew policies. This helps organizations plan for best and worst-casescenarios, and have policies ready for deployment if such scenariosarise.

The invention can be implemented to realize one or more of the followingadvantages. The logic used within the analysis is clearly exposed, suchthat the analysis results can be understood and verified by the samenon-technical personnel who understand the policies. The analysis isperformed with greater speed and ease than other methods. The policymodel used within the analysis can be used to automate policyenforcement in high-throughput transaction processing systems withoutrequiring recoding or recompilation of logic.

The details of one or more embodiments of the invention are set forth inthe accompanying drawings and the description below. Other features andadvantages of the invention will become apparent from the description,the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of a process for analyzing a business process.

FIG. 2 is a block diagram of a system for analyzing a business process.

FIG. 3 shows part of an analysis report that includes actual data andmodeled data.

FIGS. 4 and 5 show a root-cause analysis.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

In business processes, people and systems (“participants”) transforminputs (“input work items”) into operational results (“output workitems”). While work items often consist of physical materials (e.g., acar, a 100-page contract, etc.), they can be represented as a set ofvariables for the purposes of information processing, where eachvariable stores one or more values (e.g., the price of an item, aninterest rate for a loan, the origin of an order, etc.).

A business process can be modeled as a sequence of steps (“tasks”), witheach step taking zero or more input work items and producing one or moreoutput work items. Tasks are performed by participants. At the mostatomic level, tasks are either purely mechanical in nature or purelylogical in nature. In performing mechanical tasks, participants performactions—for example, painting a car, printing a document, or retrievinga record. In performing logical tasks, participants determine whatactions to perform (or what actions not to perform)—for example,determining which color to paint a car, determining which document toprint, or determining which record to retrieve. In addition, logicaltasks can act as quality controls to assess whether one or moremechanical tasks have produced the desired operational results—forexample, determining whether two or more finance managers have signed acheck over $10,000. Logical tasks are referred to synonymously as“decision points,” “business controls,” or “controls.”

Processes are governed by business policies (“policies”). Logically, apolicy can be described as a set of business rules (“rules”). Rules, asdefined by the GUIDE Business Rules Project, are statements that defineor constrain some aspect of the business. A rule is intended to assertbusiness structure, or to control or influence the behavior of thebusiness. For example, a revenue recognition policy might be describedby rules such as “An order must be shipped to customer by the lastbusiness day of quarter to qualify as revenue.”

Rules, the atomic units of policy, are enforced within businesscontrols. For example, a loan origination process may be governed by apolicy defining what products and rates to offer new loan customers. Theloan origination process may consist of the following tasks insequential order: (1) a customer fills out a loan application, (2) abroker determines what products and rates to offer the customer, (3) thebroker explains the products and rates to the customer, and (4) thecustomer selects a product. Tasks 1, 3, and 4 are mechanical tasks,while task 2 is a business control. As an example, task 2 could includea rule that states, “Applicants with good credit are eligible for homeequity lines of up to 90% of collateral value.”

As shown in FIG. 1, a process 100 is used to analyze or monitor theoperation of business controls in an existing business process. Thebusiness controls to be analyzed may be performed by people or bysystems—for example, by an enterprise software application. A work itemis input to an actual control in the business process (step 110), andthe actual control is applied to the input work item to transform itinto an output work item.

In order to analyze or monitor the business process, a modeled controlis created (step 115) that corresponds to a desired behavior of theactual control from the business process. The modeled control is modeledusing a set of business rules that logically describe the desiredbehavior, which typically is codified in a business policy. A ruleengine is used to model the control. A suitable rule engine is describedin U.S. patent application Ser. No. 09/994,477, filed Nov. 26, 2001 andin U.S. patent application Ser. No. 10/639,674, filed Aug. 11, 2003,both of which are hereby incorporated by reference in their entirety.

A modeled input work item is generated (step 120). The work item thatwas input to the actual control is used as the modeled input work itemif the work item is available (e.g., if the work item was stored beforebeing transformed into the output work item). If the work item is notavailable, but the output work item includes the information that wasinput to the actual control, the output work item is used to generate amodeled input work item that approximates the work item. The modeledcontrol is applied to the modeled input work item (step 125), and themodeled control transforms the modeled input work item into a modeledoutput work item.

As the modeled control executes on the modeled input work item, a ruleaudit trail is created (step 130) that records which rules from themodeled control fired against (that is, were applied to) the modeledinput work item, as well as the order in which the rules fired. Inaddition, the rule audit trail records the values of all variables usedas input to the rules and created as output from the rules.

The output work item from the actual business control is compared to themodeled output work item (step 140). If the output work item differsfrom the modeled output work item, an analysis engine uses the ruleaudit trail to determine which rule or rules in the modeled control werethe root causes of the differences (step 150). Once the root causes aredetermined, the actual control is changed, if necessary, to align theactual behavior of the business process more closely with the desiredbehavior, as will be discussed in more detail below.

As shown in FIG. 2, a business process 210 includes actual controls 212derived from a business policy 220. The business policy 220 specifieshow business processes and controls should operate and typicallyincludes documentation 222 codifying the business processes andcontrols. The actual controls 212 are performed by people (e.g., acompany officer signing a check to indicate approval), by automatedprocesses (e.g., software determining a loan rate based on a creditrating and a loan amount), or by a combination of the two. The actualcontrols 212 transform actual input work items 214 into actual outputwork items 216. The actual output work items 216 are stored inoperational data storage 218 (e.g., a relational database). In oneimplementation, the actual input work items 214 also are stored in theoperational data storage 218.

In order to analyze and monitor how closely the business process 210adheres to the underlying business policy 220, data from the operationaldata storage 218 is analyzed by a system and process that will bereferred to as a discrepancy analysis 230. An example of a discrepancyanalysis is a “Policy-Execution Gap Analysis,” also called a “PEGAnalysis,” both of which are trademarks of Corticon Technologies, Inc.of San Mateo, Calif. The discrepancy analysis 230 uses a data extractor232 to extract data from the operational data storage 218 and place thedata into analytical data storage 242. The extracted data in theanalytical data storage 242 is used in a modeled business process 240.The modeled business process 240 includes modeled controls 244, whichtransform modeled input work items 246 into modeled output work items248. The modeled output work items 248 are also stored in the analyticaldata storage 242. In one implementation, the data model used to definethe modeled controls 244 is identical to the data model used in theoperational data storage 218. For example, the data model used to definethe modeled controls 244 can be generated automatically based on thedata model used in the operational data storage 218, and the analyticaldata storage 242 can be generated automatically based on the data modelused to define the modeled controls 244. When the data models for themodeled controls 244 and the operational data storage 218 are identical,the data extractor 232 need only copy data from the operational datastorage 218 to the analytical data storage 242. In anotherimplementation, the data model used in the operational data storage 218differs from the data model used to define the modeled controls 244, andthe data extractor 232 must transform the data from the operational datastorage 218 into a form that is compatible with the data model used todefine the modeled controls 244. Thus, the data extractor 232 can be anextract, transform, load (ETL) software tool that can optionallytransform (reformat) the data from the operational storage 218 beforeplacing it into the analytical data storage 242.

The modeled input work items 246 correspond to the actual input workitems 214. If the actual input work items 214 are stored in theoperational data storage 218, each of the modeled work items 246contains data from a corresponding one of the actual input work items214 reformatted by the data extractor 232 into a format suitable forinput to the modeled controls 244. In some systems, not all of the datafrom the actual input work items 214 is needed by the discrepancyanalysis 230, so the modeled work items 246 may contain a subset of thedata contained in the actual input work items 214. If the actual inputwork items 214 are not stored in the operational data storage 218, butthe actual output work items 216 include the data on which the actualcontrols 212 operated, the actual output work items 216 are used as themodeled input work items 246, which then approximate the actual inputwork items 214.

The modeled controls 244 can be represented as a set of business rulesthat are defined within the business policy 220 (or within a proposedbusiness policy that differs from business policy 220). The rules may berepresented in a form similar to that described in the patentapplications incorporated by reference above or in other forms suitablefor processing by a rule engine. Each of the modeled controls 244 isautomated by a rule engine and corresponds to a portion of the businessprocess 210. Each of the modeled controls 244 implement rules that areapplied to the modeled input work items 246 to transform the modeledinput work items 246 into the modeled output work items 248. The rulescan be derived from the business policy 220 and model a desired behaviorof the actual controls 212. Each rule included in the modeled controls244 can be documented as being associated with a policy source from thebusiness policy 220 and with a motivation, thus maintaining businesscontext and traceability for the rule.

Any portion of the business process 210 involving business controls canbe modeled using the modeled controls 244 and analyzed. For businessprocess 210 to be analyzed, appropriate data about the actual outputwork items 216 must be stored in the operational data storage 218. Forexample, if a control in the business process 210 is performed by ahuman, as is the case when a check is signed, information aboutexecution of the control, such as the state of the signature on thecheck (e.g., “signed” or “unsigned”), the signatory (e.g., “Joe Smith”),and the amount (e.g., “$1,000”), typically is stored in the operationaldata storage 218 as part of the work items output from thehuman-performed control. When attempting to use the discrepancy analysis230, it may be determined that some portion of the business process 210cannot be analyzed because necessary data is not being stored in theoperational data storage 218. If this is the case, the business process210 typically can be modified so that the necessary information isstored.

When the modeled controls 244 are executed on a modeled input work item,a rule audit trail is stored in the analytical data storage 242. Therule audit trail records the sequence in which the modeled controls 244execute, which rules fire on the modeled input work item, and the orderin which the rules fire. In addition, the rule audit trail records theinput to, and the output from, each rule that fires on the modeled inputwork item.

An analysis engine 234 compares the actual output work items 216 thatare extracted to the analytical data storage 242 to the modeled outputwork items 248 to identify any differences between them. Any differenceis referred to as a discrepant work item and indicates a discrepancybetween policy and operation: a difference between an actual result anda result predicted by modeled controls. When the modeled controls 244accurately represent the business policy 220 and the modeled input workitems accurately represent the actual input work items, discrepant workitems are caused by one or both of: (1) incorrect execution (i.e.,failure to comply with policy); and (2) incorrect policy. Determinationof source requires the judgment of a subject matter expert (“SME”).

Incorrect execution is the problem when, in the judgment of the SME, therules that fired on the modeled input work items 246 to produce themodeled output work items 248 were correct. In this case, it can beinferred that the actual controls 212 failed to apply the rulesproperly, and thus failed to comply with business policy 220. Whenincorrect execution is the problem, the actual controls 212 should bemodified to perform in a manner that is compliant with the businesspolicy 220.

Incorrect policy is the problem when, in the judgment of the SME, therules that fired on the modeled input work items 246 to produce themodeled output work items 248 were incorrect. In other words, thebusiness policy 220 is wrong and should be changed.

If the actual input work items 214 are not stored in the operationaldata storage 218, a false-positive discrepant work item can exist if oneor more values of variables in the actual output work items 216 aremodified in the operational data storage 218 before the actual outputwork items 216 is extracted by the data extractor 232. A false-positivediscrepant work item indicates that the architecture of the businessprocess 210 should be modified so that the data on which the analysisrelies remains unchanged between storage and extraction. The discrepancyanalysis 230 can be used on the actual controls 212 as long as the datainput to and output from the actual controls 212 is available in theoperational data storage 218.

The analysis engine 234 uses the rule audit trail to identify the rootcause or causes of any discrepant work items and creates one or morereports 236. A root cause of a discrepant work item is a business rulethat has input values that match corresponding values in the actualinput work items 214 and output values that are different fromcorresponding values in the actual output work items 216. A root causecan be found by analyzing the rules that fired on a work item startingwith the last rule that fired. The input and output values of that rulein the modeled controls 244 are compared to the corresponding values inthe actual input work items 214 and actual output work items 216 todetermine whether that rule was a root cause. If the last rule to firewas not a root cause of the discrepancy (that is, an input to the ruleand an output from the rule both differ between the actual and modeleddata), the last rule to modify an input value that differs (thepenultimate rule) is examined. If the inputs to the penultimate rule arethe same between the actual and modeled data, the penultimate rule is aroot cause. If the inputs to the penultimate rule also differ, the lastrule to modify the penultimate rule's input that differs is examined,and so on, until a root cause is found. If multiple inputs to a givenrule differ, each last rule that modified any differing input isexamined, and the discrepant work item may have more than one rootcause. If the actual input work items 214 are not available in thediscrepancy analysis 230, the modeled input work items 246 are used intheir stead.

The reports 236 can include information about the compliance of workitems processed by individual controls or groups of controls in themodeled controls 244, can present the results of the analysis forindividual work items, and can aggregate the results of the analysisover several work items. The reports 236 allow a user to see whatpercentage of work items are discrepant. A user can determine patternsto discrepancies using the reports 236 and can identify the root causeor causes of the discrepancies by analyzing the results of theroot-cause analysis. Once the root cause is identified, the user candetermine whether the discrepant work item occurred because of problemwith the business policy 220 or because of a problem with the executionof the business policy 220.

The discrepancy analysis 230 can be used to analyze historical data fromthe business process 210 when prompted by a user or at regular timeintervals (e.g., hourly, daily, weekly, or monthly). Alternatively, thediscrepancy analysis 230 can analyze the data from the business process210 in near real time (e.g., every five seconds, or every transaction).The discrepancy analysis 230 can also analyze the data when a number ofwork items on which an actual business process has operated reaches acertain level (e.g., 100 work items).

Alerts 238 optionally are automatically generated based on the reports236. The alerts 238 indicate, for example, when violations or warningsoccur in the modeled controls 244. Violations or warnings occur when thelogic of the business rules in the modeled controls 244 is incomplete orinconsistent (that is, when the business rules do not specify outputsfor every possible input, or when they specify conflicting outputs foran input). The alerts 238 also can be generated when insufficientinformation is included in the actual output or input work items storedin the operational data storage 218. If insufficient information isincluded in the actual work items, the analysis engine 234 cannotcompare one or more values in the modeled output work items 248 to theactual output work items 216 because the actual output work items 216 donot include corresponding values. The alerts 238 also can indicate whena specific work item is discrepant, or when more than a certain numberor percentage of discrepant work items occur per unit time.

Using the reports 236, post-analysis action 250 can be performed to makethe actual output work items 216 conform to the modeled output items 248more closely. If the business process 210 is a source of differencesbetween the actual output work items 216 and the modeled output items248, the business process 210 can be modified. For example, the actualcontrols 212 can be modified by training employees, by trainingemployees differently, or by changing the operation of automatedcontrols. For some controls, an actual control can be replaced by amodeled control within the business process 210. The business process210 can also be modified by changing the content of the actual inputwork items 214 (e.g., providing additional inputs to the actual controls212). After the business process 210 is changed and new work items havebeen processed, the discrepancy analysis 230 can be used again, and theresults of the new analysis are compared to those of the old analysis toobserve the results of the change.

As a result of using the discrepancy analysis 230, it may be decidedthat the business policy 220 should be refined or redefined. If acertain rule is identified as the root cause of discrepant work items,the portion of the policy from which the rule was derived can beidentified and reconsidered.

FIG. 3 and FIG. 4 describe a discrepancy analysis performed on an ordermanagement business process, and specifically on a business control thatdetermines whether to approve or deny credit on an order. The policydocument describes five rules associated with the “approve credit”control:

Rule Number Rule Statement N.1 Company's available credit is equal tocredit limit minus credit outstanding. 1 If customer has a negative D&Brating (4-5), then do not approve credit on the order. 2 If customer hasa good payment history (average DSOs <30), then approve credit on theorder. 3 If customer has a positive D&B rating (1-3), and averageDSOs >=30, then approve credit for orders not exceeding availablecredit. 4 If order total exceeds customer's available credit, then denycredit on the order.

As shown in FIG. 3, part of the report from a discrepancy analysisincludes details of data used and created in the analysis. FIG. 3describes the actual and modeled output work items as relational tables.The business control being analyzed takes as input an order and anassociated customer. The order and customer are represented asrelational tables (“ORDER” and “CUSTOMER”), with a foreign key relatingthe tables.

The actual output work items from an actual business process aredescribed in blocks 310 and 320, and the modeled output work items aredescribed in blocks 330 and 340 and are generated from the execution ofthe modeled control. In block 330, three credit approval values (values332, 334, and 336) in the modeled output work item differ fromcorresponding credit approval values (values 312, 314, and 316) in theactual output work item. Also, one “available_credit” value 342 in the“CUSTOMER_PEG” table of block 340 (part of the modeled output work item)differs from a corresponding “available_credit” value 322 in the“CUSTOMER” table of block 320 (part of the actual output work item).

As shown in FIG. 4, the report also includes a root-cause analysis 400for the orders in FIG. 3 with discrepant output work items, where theresults differed between the historical data values and values generatedfrom the execution of the modeled control. The root-cause analysis 400shows the last rule 410 that fired on order number “3.” Examining theinputs 415 of the rule 410 shows that the actual and predicted (modeled)inputs were the same. The outputs 420 of the rule 410, however, weredifferent. This indicates that rule 410 was the root cause of thedifference in the credit approval decision.

The root-cause analysis 400 also shows two rules (rules 430 and 440)that fired on order number “5.” The rule 430 is the last rule that firedon order number “5.” The outputs 432 of the rule 430 were differentbetween the historical and modeled data, and so were the inputs 434 ofthe rule 430. Because the inputs were not the same, the rule 430 is notthe root cause of the difference in the output of the control for ordernumber “5.” The input to the rule 430 that differs between thehistorical and modeled data is the customer's available credit. Theroot-cause analysis 400 indicates that the last rule that modified theavailable credit was the rule 440. Examining the inputs 444 and outputs442 of the rule 440 shows that the outputs 442 differed, while theinputs 444 were identical. Therefore, rule 440 was the root cause of thedifference in the credit approval decision.

A discrepancy analysis can be used to analyze changes that would resultfrom a contemplated change in a business policy. For example, adiscrepancy analysis can compare the cost of historical purchases towhat those purchases would have cost if a different business policy hadgoverned the purchases. In this type of analysis, the modeled controlsused in the discrepancy analysis are based on the proposed businesspolicy rather than the actual policy that was in place when thepurchases were made. For example, this type of discrepancy analysis isused to identify opportunities for savings that a change in a purchasingpolicy would provide. As shown in FIG. 5, a root-cause analysis 500lists four rules (rules 510, 530, 550, 570) that fired when processing amodeled work item. The outputs 512 of the rule 510 indicate that adifference exists between a historical order total and the order totalthat would have resulted if an alternative business policy had governedthe processing of the work item. Examining the inputs 514 of the rule510 shows that the amounts of two of the three subtotals that were inputto the rule 510 differed between the actual results and the resultscalculated using the modeled control.

The rule 530 has different output subtotals 532, but the same inputsubtotals 534. The rule statement (“Ordered non-contracted product hasbeen replaced by ‘Exact’ or ‘Functional Equivalent’ product”) indicateswhy savings were realized when the rule 530 was applied to the workitem. Likewise, the rule 550 has different output subtotals 552 with thesame input subtotal 554. No savings were realized by the firing of therule 570. Key performance metrics 580 are included in root-causeanalysis 500 to highlight the dollar savings that would apply if the newpolicy were utilized.

In addition to the reports shown in FIGS. 3-5, additional reports can becreated from the analytical data storage 242 (FIG. 2), and theanalytical data storage 242 can be analyzed in an ad-hoc manner usinganalysis tools.

In one implementation, a discrepancy analysis is performed repeatedly tomonitor a business process with inputs whose state changes with time.For example, a business process to select an investment portfolio caninclude rules that constrain what percentage of the portfolio's totalvalue can be invested in a particular business sector. The percentage ofthe portfolio's value that is invested in the particular sectorfluctuates with time as the investments in the particular sector (andinvestments in other sectors) appreciate or depreciate in value. Adiscrepancy analysis can be run on the investment portfolio regularly(e.g., nightly) to determine whether the portfolio complies with therules given the value of the portfolio's various investments when thediscrepancy analysis is run. If the investment portfolio is no longercompliant (e.g., the percentage invested in the particular sector is toohigh or too low), a discrepant work item will occur when the discrepancyanalysis is performed, and an alert can be generated. A portfoliomanager can then reallocate the portfolio's investments to make theportfolio compliant.

The invention and all of the functional operations described in thisspecification can be implemented in digital electronic circuitry, or incomputer software, firmware, or hardware, including the structural meansdisclosed in this specification and structural equivalents thereof, orin combinations of them. The invention can be implemented as one or morecomputer program products, i.e., one or more computer programs tangiblyembodied in an information carrier, e.g., in a machine-readable storagedevice or in a propagated signal, for execution by, or to control theoperation of, data processing apparatus, e.g., a programmable processor,a computer, or multiple processors or computers. A computer program(also known as a program, software, software application, or code) canbe written in any form of programming language, including compiled orinterpreted languages, and it can be deployed in any form, including asa stand-alone program or as a module, component, subroutine, or otherunit suitable for use in a computing environment. A computer programdoes not necessarily correspond to a file. A program can be stored in aportion of a file that holds other programs or data, in a single filededicated to the program in question, or in multiple coordinated files(e.g., files that store one or more modules, sub-programs, or portionsof code). A computer program can be deployed to be executed on onecomputer or on multiple computers at one site or distributed acrossmultiple sites and interconnected by a communication network.

The processes and logic flows described in this specification, includingthe method steps of the invention, can be performed by one or moreprogrammable processors executing one or more computer programs toperform functions of the invention by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus of the invention can be implemented as, specialpurpose logic circuitry, e.g., an FPGA (field programmable gate array)or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for executing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto-optical disks, or optical disks. However, a computerneed not have such devices. Moreover, a computer can be embedded inanother device, e.g., a mobile telephone, a personal digital assistant(PDA), a mobile audio player, a Global Positioning System (GPS)receiver, to name just a few. Information carriers suitable forembodying computer program instructions and data include all forms ofnon-volatile memory, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, the invention can be implementedon a computer having a display device, e.g., a CRT (cathode ray tube) orLCD (liquid crystal display) monitor, for displaying information to theuser and a keyboard and a pointing device, e.g., a mouse or a trackball,by which the user can provide input to the computer. Other kinds ofdevices can be used to provide for interaction with a user as well; forexample, feedback provided to the user can be any form of sensoryfeedback, e.g., visual feedback, auditory feedback, or tactile feedback;and input from the user can be received in any form, including acoustic,speech, or tactile input.

The invention can be implemented in a computing system that includes aback-end component, e.g., as a data server, or that includes amiddleware component, e.g., an application server, or that includes afront-end component, e.g., a client computer having a graphical userinterface or a Web browser through which a user can interact with animplementation of the invention, or any combination of such back-end,middleware, or front-end components. The components of the system can beinterconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication networksinclude a local area network (“LAN”) and a wide area network (“WAN”),e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

Certain features which, for clarity, are described in this specificationin the context of separate embodiments, may also be provided incombination in a single embodiment. Conversely, various features which,for brevity, are described in the context of a single embodiment, mayalso be provided in multiple embodiments separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

The invention has been described in terms of particular embodiments.Other embodiments are within the scope of the following claims. Forexample, the steps of the invention can be performed in a differentorder and still achieve desirable results.

What is claimed is:
 1. A system comprising: one or more processors; and a memory storing instructions, when executed by the one or more processor, to provide: an analysis engine operable to compare one or more actual output work items generated by participants of an actual business process representing a business policy to one or more modeled output work items from one or more modeled business controls representing the business policy and to report differences between the one or more actual output work items and the one or more modeled output work items indicating how closely the actual business process adheres to the business policy, wherein: each actual output work item includes one or more actual output data values, the actual business process changing an actual input work item into the actual output work item; each modeled output work item includes one or more modeled output data values, the one or more modeled business controls executing on one or more modeled input work items and generating the one or more modeled output work items; input data values included in each modeled input work item are determined from one of matching (1) corresponding values included in a corresponding actual input work item and (2) the corresponding actual output work item; each modeled business control is defined by a set of business rules that represent the business policy, the analysis engine being operable to use the set of business rules to generate the one or more modeled output work items using the input data values of the one or more modeled input work items; responsive to determining that one of the one or more actual output work items differs from a corresponding modeled output work item, the analysis engine creates a rule audit trail for identifying a root cause of a discrepant work item by analyzing the last business rule from the set of business rules that have been used to generate the corresponding modeled output work item, the discrepant work item indicating a difference between the one of the one or more actual output work items and the corresponding modeled output work item, the root cause including a business rule from the set of business rules, the root cause: having input data values matching to the corresponding values included in the one of the one or more actual input work items; having modeled output data values that are different from the corresponding values in the one of the one or more actual output work items; and indicating that the discrepant work item is caused by one of: an incorrect execution of the business policy; and an incorrectness in the business policy.
 2. The system of claim 1, wherein: the rule audit trail records, for an individual modeled output work item, a sequence of modeled business controls that executed on an individual modeled input work item and a sequence of associated business rules that fired within the modeled business controls in the sequence of modeled business controls to generate the individual modeled output work item.
 3. The system of claim 1, wherein: the analysis engine is operable to compare the one or more actual output work items and the one or more modeled output work items in near real-time and generate an alert if a difference is found between the one or more actual output work items and the one or more modeled output work items.
 4. The system of claim 1, wherein: the analysis engine is operable to aggregate results of comparing the one or more actual output work items and the one or more modeled output work items to report the differences.
 5. The system of claim 1, wherein: the analysis engine is operable to generate an alert when a discrepant work item occurs.
 6. The system of claim 1, wherein: the analysis engine is operable to compare the one or more actual output work items to one or more updated modeled output work items from one or more updated modeled business controls and report a change caused by using the one or more updated modeled business controls in place of the one or more modeled business controls.
 7. The system of claim 1, wherein: an analytical data storage database stores each modeled input work item and each modeled output work item, and the analytical data storage database is constructed automatically based upon a data model used to define the one or more modeled business controls.
 8. The system of claim 1, wherein: a data model is used to construct the one or more modeled business controls, and the data model is constructed automatically based upon an operational data model used within an operational data storage that stores each actual output work item and each actual input work item.
 9. The system of claim 1, wherein: the analysis engine is operable to generate an alert if the set of business rules that defines one of the modeled business controls is incomplete or inconsistent.
 10. The system of claim 1, wherein: the analysis engine is operable to generate an alert if one of the actual output work items does not include any actual output data value that corresponds to a modeled output data value included in one of the modeled output work items.
 11. The system of claim 2, wherein: the rule audit trail also records a set of data values input to each business rule that fired and a set of data values output from each business rule that fired.
 12. The system of claim 2, wherein: the root cause causes a difference between the individual modeled output work item and the corresponding actual output work item.
 13. A system comprising: one or more processors and a memory storing instructions, when executed by the one or more processor, to perform operations including: comparing one or more actual output work items generated by participants of an actual business process representing a business policy to one or more modeled output work items from a modeled business process representing the business policy to determine how closely the actual business process adheres to the business policy; and reporting differences between the one or more actual output work items and the one or more modeled output work items, wherein: each actual output work item includes one or more actual output data values, the actual business process changing an actual input work item into the actual output work item; each modeled output work item includes one or more modeled output data values, the modeled business process operating on one or more modeled input work items and generating the one or more modeled output work items; input data values included in each modeled input work item are determined from one of matching (1) corresponding values included in a corresponding actual input work item and (2) the corresponding actual output work item; the modeled business process includes one or more modeled business controls, each modeled business control corresponding to an actual business control included in the actual business process; each modeled business control is defined by a set of business rules that represent the business policy, the means for comparing being operable to use the set of business rules to generate the one or more modeled output work item using the input data values of the one or more modeled input work items; and responsive to determining that one of the one or more actual output work items differs from a corresponding modeled output work item, creating a rule audit trail for identifying a root cause of a discrepant work item by analyzing the last business rule from the set of business rules that have been used to generate the corresponding modeled output work item, the discrepant work item indicating a difference between the one of the one or more actual output work items and the corresponding modeled output work item, the root cause including a business rule from the set of business rules, the root cause: having input data values matching to the corresponding values included in the one of the one or more actual input work item; having modeled output data values that are different from the corresponding values in the one of the one or more actual output work items; and indicating that the discrepant work item is caused by one of: an incorrect execution of the business policy; and an incorrectness in the business policy.
 14. The system of claim 13, wherein: the rule audit trail records, for an individual modeled output work item, a sequence of modeled business controls that executed on an individual modeled input work item and a sequence of associated business rules that fired within the modeled business controls in the sequence of modeled business controls to generate the individual modeled output work item.
 15. A method comprising: executing, using one or more processors, one or more modeled business controls on a modeled input work item to generate a corresponding modeled output work item, wherein input data values included in the modeled input work item are determined from one of matching (1) corresponding values included in a corresponding actual input work item and (2) the corresponding actual output work item, each modeled business control is defined by a set of business rules that represent a business policy, and executing the one or more modeled business controls includes using the set of business rules and the input data values of the modeled input work item to generate the modeled output work item; comparing an actual output work item from an actual business process representing the business policy to the modeled output work item from the one or more modeled business controls representing the business policy, wherein the actual output work item includes one or more actual output data values, the actual business process changes the actual input work item into the actual output work item, and the modeled output work item includes one or more modeled output data values; reporting differences between the actual output work item and the modeled output work item indicating how closely the actual business process adheres to the business policy; and responsive to determining that one of the one or more actual output work items differs from a corresponding modeled output work item, creating a rule audit trail for identifying a root cause of a discrepant work item by analyzing the last business rule from the set of business rules that have been used to generate the corresponding modeled output work item, the discrepant work item indicating a difference between the one of the one or more actual output work items and the corresponding modeled output work item, the root cause including a business rule from the set of business rules, the root cause: having input data values matching to the corresponding values included in the one of the one or more actual input work items; having modeled output data values that are different from the corresponding values in the one of the one or more actual output work items; and indicating that the discrepant work item is caused by one of: an incorrect execution of the business policy; and an incorrectness in the business policy.
 16. The method of claim 15, wherein: the rule audit trail records, for the modeled output work item, a sequence of the modeled business controls that executed on the modeled input work item and a sequence of the business rules that fired within the modeled business controls to generate the modeled output work item.
 17. The method of claim 16, further comprising: recording a set of data values input to each business rule that fired and a set of data values output from each business rule that fired.
 18. The method of claim 16, wherein: the root cause causes a difference between the individual modeled output work item and the actual output work item.
 19. The method of claim 15, further comprising: comparing the actual output work item to an updated modeled output work item from one or more updated modeled business controls; and reporting a change caused by using the one or more updated modeled business controls in place of the one or more modeled business controls.
 20. The method of claim 15, further comprising: creating the set of business rules by modeling logic of the business policy.
 21. A computer program product, tangibly embodied in a machine-readable storage device, the computer program product comprising instructions operable to cause a data processing apparatus to perform the operations of: executing one or more modeled business controls on a modeled input work item to generate a corresponding modeled output work item, wherein input data values included in the modeled input work item are determined from one of matching (1) corresponding values included in a corresponding actual input work item and (2) the corresponding actual output work item, each modeled business control is defined by a set of business rules that represent a business policy, and executing the one or more modeled business controls includes using the set of business rules and the input data values of the modeled input work item to generate the modeled output work item; comparing an actual output work item from an actual business process representing the business policy to the modeled output work item from the modeled business controls representing the business policy, wherein the actual output work item includes one or more actual output data values, the actual business process changes the actual input work item into the actual output work item, and the modeled output work item includes one or more modeled output data values; reporting differences between the actual output work item and the modeled output work item, the differences indicating differences between the actual business process and the business policy; and responsive to determining that one of the one or more actual output work items differs from a corresponding modeled output work item, creating a rule audit trail for identifying a root cause of a discrepant work item by analyzing the last business rule from the set of business rules that have been used to generate the corresponding modeled output work item, the discrepant work item indicating a difference between the one of the one or more actual output work items and the corresponding modeled output work item, the root cause including a business rule from the set of business rules, the root cause: having input data values matching to the corresponding values included in the one of the one or more actual input work items; having modeled output data values that are different from the corresponding values in the one of the one or more actual output work items; and indicating that the discrepant work item is caused by one of: an incorrect execution of the business policy; and an incorrectness in the business policy.
 22. The computer program product of claim 21, wherein: the rule audit trail records, for the modeled output work item, a sequence of the modeled business controls that executed on the modeled input work item and a sequence of the business rules that fired within the modeled business controls to generate the modeled output work item.
 23. The computer program product of claim 22, further comprising instructions operable to cause a data processing apparatus to perform the operation of: recording a set of data values input to each business rule that fired and a set of data values output from each business rule that fired.
 24. The computer program product of claim 22, wherein: the root cause causes a difference between the individual modeled output work item and the actual output work item.
 25. The computer program product of claim 21, further comprising instructions operable to cause a data processing apparatus to perform the operations of: comparing the actual output work item to an updated modeled output work item from one or more updated modeled business controls; and reporting a change caused by using the one or more updated modeled business controls in place of the one or more modeled business controls.
 26. A system for comparing outputs of internal business controls to outputs of business controls modeling a policy, the system comprising: one or more processors and a memory storing instructions, when executed by the one or more processor, to provide: an analysis engine operable to compare one or more actual output work items generated from participants of one or more actual business controls in an actual business process representing a business policy to one or more modeled output work items from one or more modeled business controls representing the business policy, wherein: each actual output work item includes a set of variables, each variable storing one or more values, the one or more actual business controls executing on and changing an actual input work item into the actual output work item; each modeled output work item includes a set of variables, each variable storing one or more values, the one or more modeled business controls executing on and changing a modeled input work item into the modeled output work item; each actual business control is executed by an existing system or manually by one or more people, and each modeled business control is executed by a rule engine; each modeled business control includes a set of business rules that describe a desired logic for the modeled business control, the set of business rules operable to transform the modeled input work item into the modeled output work item; each business rule in the set of business rules is associated with a policy source and a motivation; each modeled input work item is extracted from operational data stores that store the one or more actual output work items; each modeled input work item is determined from one of matching (1) a corresponding actual input work item and (2) the corresponding actual output work item; the one or more modeled business controls each fire one or more business rules from the set of business rules included in the respective modeled business control; the analysis engine creates a rule audit trail and stores the rule audit trail in an analytical data store, the rule audit trail recording, for an individual modeled output work item, a sequence of modeled business controls and associated business rules that fired against a corresponding modeled input work item, a set of variable inputs to each business rule that fired, and a set of variable outputs from each business rule that fired; the analysis engine uses the rule audit trails to identify any differences between the actual output work items and corresponding modeled output work items indicating how closely the actual business process adheres to the business policy; and for any difference between the actual output work items and the corresponding modeled output work items, the analysis engine uses the rule audit trails to identify a root cause of a discrepant work item by analyzing the last business rule from the set of business rules that have been used to generate the corresponding modeled output work item, the discrepant work item indicating a difference between the one of the one or more actual output work items and the corresponding modeled output work item and generates reports, the root cause including a business rule from the set of business rules: having input data values matching to the corresponding values included in the one of the one or more actual input work items; having modeled output data values that are different from the corresponding values in the one of the one or more actual output work items; and indicating that the discrepant work item is caused by one of: an incorrect execution of the business policy; and an incorrectness in the business policy.
 27. The system of claim 26, wherein: the desired logic for each modeled business control is derived from the policy; the policy is an existing policy; and any differences between the actual output work items and the corresponding modeled output work items are discrepancies between the policy and an execution of the policy.
 28. The system of claim 26, wherein: the desired logic for each modeled business control is derived from the policy; the policy is a proposed new policy; and any differences between the actual output work items and the corresponding modeled output work items reflect the effects of changing from an existing policy to the proposed new policy. 